proxy: add support for anthropic v1/messages api#417
Conversation
WalkthroughRenamed the request handler in proxy from Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
proxy/process.go (1)
510-518: Clarify the debug log to reflect all gating conditions for the loading-state SSEThe
isChatCompletionsguard correctly limits OAI-shaped loading-state SSE to/v1/chat/completions, which is appropriate for Anthropic/v1/messagescompatibility. However, the else-branch log fires for three different conditions (SendLoadingState unset/false, non-streaming, or non-chat-completions path) but only mentions SendLoadingState, making debugging harder.Update the log message to reflect all gating conditions:
- p.proxyLogger.Debugf("<%s> SendLoadingState is nil or false, not streaming loading state", p.ID) + p.proxyLogger.Debugf( + "<%s> loading-state streaming disabled (SendLoadingState=%v, streaming=%v, isChatCompletions=%v)", + p.ID, + p.config.SendLoadingState != nil && *p.config.SendLoadingState, + isStreaming, + isChatCompletions, + )Per project guidelines, run
make test-devafter this change to verify static analysis and tests pass.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
proxy/process.go(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
proxy/**/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
Run
make test-devwhen making iterative changes to code under theproxy/directory - this runsgo testandstaticcheck, and all static checking errors must be fixed
Files:
proxy/process.go
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: run-tests
1 participant
llama.cpp now supports the anthropic /v1/messages api (ggml-org/llama.cpp#17570). This commit adds support for it in llama-swap.
Since llama-swap only needs the
modelparameter this was a fairly trivial addition.Summary by CodeRabbit
Refactor
Bug Fix
✏️ Tip: You can customize this high-level summary in your review settings.